Goto

Collaborating Authors

 Charlotte County


SCOP: Evaluating the Comprehension Process of Large Language Models from a Cognitive View

Xiao, Yongjie, Liang, Hongru, Qin, Peixin, Zhang, Yao, Lei, Wenqiang

arXiv.org Artificial Intelligence

Despite the great potential of large language models(LLMs) in machine comprehension, it is still disturbing to fully count on them in real-world scenarios. This is probably because there is no rational explanation for whether the comprehension process of LLMs is aligned with that of experts. In this paper, we propose SCOP to carefully examine how LLMs perform during the comprehension process from a cognitive view. Specifically, it is equipped with a systematical definition of five requisite skills during the comprehension process, a strict framework to construct testing data for these skills, and a detailed analysis of advanced open-sourced and closed-sourced LLMs using the testing data. With SCOP, we find that it is still challenging for LLMs to perform an expert-level comprehension process. Even so, we notice that LLMs share some similarities with experts, e.g., performing better at comprehending local information than global information. Further analysis reveals that LLMs can be somewhat unreliable -- they might reach correct answers through flawed comprehension processes. Based on SCOP, we suggest that one direction for improving LLMs is to focus more on the comprehension process, ensuring all comprehension skills are thoroughly developed during training.


Urban-STA4CLC: Urban Theory-Informed Spatio-Temporal Attention Model for Predicting Post-Disaster Commercial Land Use Change

Guo, Ziyi, Wang, Yan

arXiv.org Artificial Intelligence

Natural disasters such as hurricanes and wildfires increasingly introduce unusual disturbance on economic activities, which are especially likely to reshape commercial land use pattern given their sensitive to customer visitation. However, current modeling approaches are limited in capturing such complex interplay between human activities and commercial land use change under and following disturbances. Such interactions have been more effectively captured in current resilient urban planning theories. This study designs and calibrates a Urban Theory-Informed Spatio-Temporal Attention Model for Predicting Post-Disaster Commercial Land Use Change (Urban-STA4CLC) to predict both the yearly decline and expansion of commercial land use at census block level under cumulative impact of disasters on human activities over two years. Guided by urban theories, Urban-STA4CLC integrates both spatial and temporal attention mechanisms with three theory-informed modules. Resilience theory guides a disaster-aware temporal attention module that captures visitation dynamics. Spatial economic theory informs a multi-relational spatial attention module for inter-block representation. Diffusion theory contributes a regularization term that constrains land use transitions. The model performs significantly better than non-theoretical baselines in predicting commercial land use change under the scenario of recurrent hurricanes, with around 19% improvement in F1 score (0.8763). The effectiveness of the theory-guided modules was further validated through ablation studies. The research demonstrates that embedding urban theory into commercial land use modeling models may substantially enhance the capacity to capture its gains and losses. These advances in commercial land use modeling contribute to land use research that accounts for cumulative impacts of recurrent disasters and shifts in economic activity patterns.


In These Small Cities, AI Advances Could Be Costly

MIT Technology Review

It's long been clear that urbanization and automated technologies are shaping society, but it hasn't been obvious how the two forces affect each other. A new study from MIT's Media Lab posits that the smaller the city, the greater the impact it faces from automation. The finding, they say, could encourage legislators to pay special attention to workers in smaller cities and offer them support services. Other researchers have attempted to measure the effect of technology on employment in cities, but the Media Lab authors, who have identified which jobs and skills tend to be more prevalent in smaller cities and larger ones, claim to be the first to explain why different U.S. cities are more susceptible (or resilient) to technological unemployment. They say that bigger cities have a disproportionately large number of jobs for people who do cognitive and analytical tasks, such as software developers and financial analysts--occupations that are less likely to be disrupted by automation.